Method And System For Processing An Image

ABSTRACT

The present invention includes a method and system for processing images captured with an image-capturing device. According to the present invention, a method and system includes reconfiguring a display of an image based on the orientation of the image-capturing device when the image is captured. Through the use of the method and system in accordance with the present invention, a user can view captured images without having to account for a rotation of the image-capturing device. The method and system includes capturing the image with an image-capturing device, determining an orientation of the image-capturing device and reconfiguring a display of the image based on the orientation of the image-capturing device.

FIELD OF THE INVENTION

The present invention relates to the field of digital cameras, and moreparticularly relates to a method and system for processing a capturedimage.

BACKGROUND OF THE INVENTION

In digital cameras, images are represented by data and stored either inthe camera's memory or an external memory device from which they can beaccessed by a user. A significant advantage to digital cameras is thatusers then have the capability to manipulate the image data in a numberof ways. Users are able to operate on and modify the images, transferthem to other devices, incorporate them into documents, display them ina variety of formats, and the like. Thus, in comparison to conventionalcameras, digital cameras introduce a variety of capabilities andenhancements.

The digital camera incorporates a central processing unit, memory, andmany other features of a computer system. Accordingly, the digitalcamera is capable of concurrently running multiple software routines andsubsystems to control and coordinate the various processes of thecamera. One subsystem of particular interest is the image processingsubsystem that is used for analyzing and manipulating captured imagedata in a variety of ways, including linearization, defect correction,white balance, interpolation, color correction, image sharpening, andcolor space conversion. In addition, the subsystem typically coordinatesthe functioning and communication of the various image processing stagesand handles the data flow between the various stages.

Most digital cameras today are similar in size to and behave likeconventional point-and-shoot cameras. Unlike conventional cameras,however, most digital cameras store digital images in an internal flashmemory or on external memory cards, and some are equipped with aliquid-crystal display (LCD) screen on the back of the camera. Throughthe use of the LCD, most digital cameras operate in two modes, recordand play, although some only have a record mode. In record mode, the LCDis used as a viewfinder in which the user may view an object or scenebefore taking a picture. In play mode, the LCD is used as a playbackscreen for allowing the user to review previously captured images eitherindividually or in arrays of four, nine, or sixteen images. Digitalcameras can typically be coupled with a peripheral display, such as atelevision set or a computer display such as a printer or the like. Inthis manner, the user may view the various images stored within thedigital camera on a larger display.

A problem exists however when a user attempts to view an image capturedwith the camera. Sometimes a camera user will rotate the camera, whencapturing an image, in order to get a “full length” or “portrait” shotof the image being captured. This presents a problem when it comes timeto view the image on the LCD screen of the camera because the user nowhas to rotate the camera in order to properly view the image.Additionally, if the image is exported to a peripheral display, such asa television set or a computer display, the user can't properly view thecaptured image, without rotating her head in an uncomfortable fashion ormanually manipulating the image with image processing software.

Accordingly, what is needed is a method and system that allows a digitalcamera user to view captured images without having to rotate her head ormanually manipulate the captured image. The method and system should besimple, cost effective and capable of being easily adapted to existingtechnology. The present invention addresses these needs.

SUMMARY OF THE INVENTION

The present invention includes a method/system for processing imagescaptured with an image-capturing device. According to variousembodiments of the present invention, a method/system includesreconfiguring a display of an image based on the orientation of theimage-capturing device when the image is captured. Through the use ofthe method and system in accordance with the present invention, a usercan view captured images without having to account for a rotation of theimage-capturing device.

A first aspect of the present invention includes a method of processingan image. The method includes capturing the image with animage-capturing device, determining an orientation of theimage-capturing device and reconfiguring a display of the image based onthe orientation of the image-capturing device.

A second aspect of the present invention includes an image processingsystem. The image processing system includes an image-capturing devicecapable of determining an orientation thereof in relation to a referenceaxis and a system coupled to the image-capturing device wherein thesystem is capable of displaying the captured image based on theorientation of the image-capturing device when the image is captured.

Other aspects and advantages of the present invention will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, illustrating by way of example theprinciples of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level flow chart of a method in accordance with anembodiment of the present invention.

FIG. 2 is a block diagram of an image-capturing device according to anembodiment of the present invention.

FIG. 3 is a more detailed block diagram of an imaging device accordingto an embodiment of the present invention.

FIG. 4 a more detailed block diagram of an internal computer accordingto an embodiment of the present invention.

FIG. 5A shows an image-capturing device in a normal position to capturean image of object in accordance with an embodiment of the presentinvention.

FIG. 5B shows the image-capturing device in a rotated orientation withrespect to a reference axis in accordance with an embodiment of thepresent invention.

FIG. 6 shows an image processing system in accordance with an embodimentof the present invention.

FIG. 7 an illustration of a PC that can be utilized in conjunction withthe image processing system in accordance with an embodiment of thepresent invention.

FIG. 8 is a flowchart of a method in accordance with an alternateembodiment of the present invention.

DETAILED DESCRIPTION

The present invention relates to a method and system for processing animage. The following description is presented to enable one of ordinaryskill in the art to make and use the invention and is provided in thecontext of a patent application and its requirements. Variousmodifications to the preferred embodiment and the generic principles andfeatures described herein will be readily apparent to those skilled inthe art. Thus, the present invention is not intended to be limited tothe embodiment shown but is to be accorded the widest scope consistentwith the principles and features described herein.

The present invention includes a method and system for processing imagescaptured with an image-capturing device. According to the presentinvention, a method and system includes reconfiguring a display of animage based on the orientation of the image-capturing device when theimage is captured. Through the use of the method and system inaccordance with the present invention, a user can view captured imageswithout having to account for a rotation of the image-capturing devicewhen the image is captured.

For a further understanding of the present invention, please refer nowto FIG. 1. FIG. 1 is a flowchart of a method in accordance with anembodiment of the present invention. A first step 110 includes capturingan image. The next step 120 includes determining an orientation of theimage-capturing device. A final step 130 includes reconfiguring adisplay of the image based on the orientation. This step can includereconfiguring the captured image on a display screen of theimage-capturing device or reconfiguring the captured image on aperipheral display coupled to the image-capturing device.

In an embodiment, step 110 is accomplished utilizing an image-capturingdevice such as a digital camera or the like. For an example of such adevice, please refer to FIG. 2. FIG. 2 is a block diagram of animage-capturing device 200 in accordance with an embodiment of thepresent invention. Image-capturing device 200 can include an imagingdevice 202, a system bus 204, a computer 206 and an orientationdetection mechanism 208. Imaging device 202 can be optically coupled toan object 201 and electrically coupled via system bus 204 to computer206.

The orientation detection mechanism 208 can be coupled to the imagingdevice 202. Once a photographer has focused imaging device 202 on object201 and, using a capture button or some other means, instructedimage-capturing device 200 to capture an image of object 201, computer206 can command the imaging device 202 via system bus 204 to capture rawimage data representing object 201. The captured raw image data can betransferred over system bus 204 to computer 206 which performs variousimage processing functions on the image data before storing it in itsinternal memory. System bus 204 also passes various status and controlsignals between imaging device 202 and computer 206.

Although the above image-capturing device of the present invention isdescribed in the context of being a digital camera, one of ordinaryskill in the art will readily recognize that the image-capturing devicecan be a mobile phone, a personal-digital-assistant (PDA) or a varietyof other devices, while remaining within the spirit and scope of thepresent invention.

Referring now to FIG. 3, a more detailed block diagram of an embodimentof the imaging device 202 is shown. Imaging device 202 can be a lens 220having an iris, a filter 222, an image sensor 224, a timing generator226, an analog signal processor (ASP) 228, an analog-to-digital (A/D)converter 230, an interface 232, and one or more motor 234.

In operation, imaging device 202 captures an image of object 201 viareflected light impacting image sensor 224 along optical path 236. Imagesensor 224 responsively generates a set of raw image data representingthe captured image. The raw image data can then be routed through ASP228, A/D converter 230 and interface 232. Interface 232 has outputs forcontrolling ASP 228, motors 234 and timing generator 226. From interface232, the raw image data passes over system bus 204 to the internalcomputer 206.

Referring now to FIG. 4, a more detailed block diagram of an embodimentof the internal computer 206 is shown. System bus 204 providesconnection paths between imaging device 202, power manager 342, centralprocessing unit (CPU) 344, random-access memory (DRAM, MRAM, FeRAM,etc.) 346, input/output interface (I/O) 348, read-only memory (ROM) 350,and buffers/connector 352. Removable memory 354 connects to system bus204 via buffers/connector 352. Alternately, image-capturing device 200may be implemented without removable memory 354 or buffers/connector352.

Power manager 342 communicates via line 366 with power supply 356 andcoordinates power management operations for image-capturing device 200.CPU 344 typically includes a conventional processor device forcontrolling the operation of image-capturing device 200. In anembodiment, CPU 344 can be capable of concurrently running multiplesoftware routines to control the various processes of image-capturingdevice 200 within a multi-threading environment. RAM 346 can be acontiguous block of dynamic memory which may be selectively allocated tovarious storage functions.

I/O 348 can be an interface device allowing communications to and fromcomputer 206. For example, I/O 348 permits an external host computer(not shown) to connect to and communicate with computer 206. I/O 348also permits an image-capturing device 200 user to communicate withimage-capturing device 200 via an external user interface and via anexternal display panel. This is referred to as a view finder.

ROM 350 can include a nonvolatile read-only memory which stores a set ofcomputer-readable program instructions to control the operation ofimage-capturing device 200. Removable memory 354 serves as an additionalimage data storage area and is preferably a non-volatile device, readilyremovable and replaceable by a image-capturing device 200 user viabuffers/connector 352. Thus, a user who possesses several removablememories 354 may replace a full removable memory 354 with an emptyremovable memory 354 to effectively expand the picture-taking capacityof image-capturing device 200. In an embodiment of the presentinvention, removable memory 354 is typically implemented using a flashdisk.

Power supply 356 supplies operating power to the various components ofimage-capturing device 200. In an embodiment, power supply 356 providesoperating power to a main power bus 362 and also to a secondary powerbus 364. The main power bus 362 provides power to imaging device 202,I/O 348, ROM 350 and removable memory 354. The secondary power bus 364provides power to power manager 342, CPU 344 and RAM 346.

Power supply 356 can be connected to main batteries 358 and also tobackup batteries 360. In an embodiment, a user of the image-capturingdevice 200 may also connect power supply 356 to an external powersource. During normal operation of power supply 356, the main batteries358 provide operating power to power supply 356 which then provides theoperating power to image-capturing device 200 via both main power bus362 and secondary power bus 364.

During a power failure mode in which the main batteries 358 have failed,the backup batteries 360 provide operating power to power supply 356which then provides the operating power only to the secondary power bus364 of image-capturing device 200. Selected components ofimage-capturing device 200 (including RAM 346) are thus protectedagainst a power failure in main batteries 358.

Power supply 356 can also include a flywheel capacitor connected to thepower line coming from the main batteries 358. If the main batteries 358suddenly fail, the flywheel capacitor temporarily maintains the voltagefrom the main batteries 358 at a sufficient level, so that computer 206can protect any image data currently being processed by image-capturingdevice 200 before shutdown occurs.

Referring back to FIG. 1, in an embodiment, step 120 can be accomplishedutilizing an orientation detection mechanism coupled within theimage-capturing device 200. The orientation detection mechanism 208 iscapable of determining the orientation of the image-capturing device 200at the time the image-capturing device 200 captures an image of anobject. For a better understanding, please refer to FIGS. 5A and 5B.FIG. 5A shows the image-capturing device 200, including the orientationdetection mechanism 208, in a normal position to capture an image of anobject 201. Also, shown on the image-capturing device 200 is an LCDscreen 402. In the normal position, the orientation of theimage-capturing device 200 is level with respect to a reference axis. Inan embodiment, the reference axis is the horizontal plane 205.

When the orientation of the image-capturing device 200 is level(±5-10°), the image of the object 201 is captured by the image-capturingdevice 200 and stored as an image file in a regular fashion. However, ifthe orientation of the image-capturing device 200 is not level withrespect to the reference axis 205, then the orientation detectionmechanism 208, determines the orientation of the image-capturing device200 with respect to the reference axis 205 and stores this informationin the image file of the captured image. In an embodiment, theorientation of the image-capturing device 200 is stored in the imagefile header. A file header is the first part of the image file andcontains controlling data as well as the structural layout of thecontents of the image file.

For a better understanding, please refer to FIG. 5B. FIG. 5B shows theimage-capturing device 200 in a rotated orientation with respect to areference axis 205. As can be seen in FIG. 5B, the image-capturingdevice 200 has been rotated approximately 90° to the left. Accordingly,the orientation detection mechanism 208, determines the orientation ofthe image-capturing device 200 with respect to the reference axis 205and subsequently stores this orientation information in the associatedimage file. Once the orientation information is stored in the associatedimage file, the resulting display of the image can be reconfigured inorder to account for the orientation of the image-capturing device 200with respect to the reference axis 205.

The image file is accordingly processed with image processing softwarewhereby the image is reconfigured based on the orientation of theimage-capturing device 200 with respect to the reference axis andsubsequently displayed as though the orientation of the image-capturingdevice 200 was level with respect to the reference axis. In anembodiment, the image is reconfigured utilizing “local” image processingsoftware contained within the internal computer 206 of theimage-capturing device 200. For example, if the orientation detectionmechanism 208 determines that the orientation of the image-capturingdevice 200 is 90° from the reference axis 205, the orientation is storedwithin the image file and local image processing software will rotate(re-orient) the captured image whereby the captured image is displayedas though the image-capturing device 200 was level at the time the imagewas captured.

An embodiment of the orientation detection mechanism 208 includes anaccelerometer. An accelerometer, as is well known to those skilled inthe art, detects acceleration and provides a voltage output that isproportional to the detected acceleration. Most accelerometers measureacceleration based on the effect gravity has on the desired object. Bydetecting the direction of the gravitational acceleration, theaccelerometer can be utilized to sense the orientation of theimage-capturing device. The orientation information can then be storedin an associated image file.

Although the above-described embodiment of the present invention isdescribed in the context of being implemented in conjunction with anaccelerometer, one of ordinary skill in the art will readily recognizethat a variety of devices can be utilized to determine the orientationof the image-capturing device 200 with respect to the reference axiswhile remaining within the spirit and scope of the present invention.

Additionally, in an embodiment of the present invention, the orientationdetection capability of the image-capturing device 200 can be turned onor off by the user. Accordingly, the image-capturing device 200 canoperate in “re-orientation” mode whereby the orientation detectionmechanism 208 determines the orientation of the image-capturing device200 with respect to a reference axis when an image is captured and theorientation information is stored in the image file. Alternatively, theuser can disable the orientation detection capability of theimage-capturing device 200 via a switch, button, on board menu or othermeans, thus operating the image-capturing device 200 in a “normal” modewhereby image files are created without including the orientationinformation.

An alternate embodiment of the present invention includes an imageprocessing system. As shown in FIG. 6, the image processing system 600includes an image-capturing device 605 (similar to image-capturingdevice 200), a personal computer system (PC) 610, and a printer 615. Theimage-capturing device 605 and PC 610 can be connected to each other viaa communication cable 620 such as an RS232C cable and the PC 610 andprinter 615 are connected to each other via a communication cable 625such as a Centronics cable. Referring back to FIG. 6, the imageprocessing system 600 can include a PC 610.

For an example of such a PC, please refer now to FIG. 7. FIG. 7 is anillustration of a PC 610 that can be utilized in conjunction with theimage processing system 600. The PC 610, including, a keyboard 611, amouse 612 and a printer 615 are depicted in block diagram form. The PC610 includes a system bus or plurality of system buses 621 to whichvarious components are coupled and by which communication between thevarious components is accomplished. The microprocessor 622 is connectedto the system bus 621 and is supported by read only memory (ROM) 623 andrandom access memory (RAM) 624 also connected to the system bus 621. Amicroprocessor is one of the Intel family of microprocessors includingthe 386, 486 or Pentium microprocessors. However, other microprocessorsincluding, but not limited to, Motorola's family of microprocessors suchas the 68000, 68020 or the 68030 microprocessors and various ReducedInstruction Set Computer (RISC) microprocessors such as the PowerPC chipmanufactured by IBM. Other RISC chips made by Hewlett Packard, Sun,Motorola and others may be used in the specific computer.

The ROM 623 contains, among other code, the Basic Input-Output system(BIOS) which controls basic hardware operations such as the interactionof the processor and the disk drives and the keyboard. The RAM 624 isthe main memory into which the operating system 640 and remote imageprocessing software 650 are loaded. (Image processing software 650 iscategorized as “remote” in that it is not contained withinimage-capturing device 605.) The memory management chip 625 is connectedto the system bus 621 and controls direct memory access operationsincluding, passing data between the RAM 624 and hard disk drive 626 andfloppy disk drive 627. The CD ROM 632 also coupled to the system bus 621is used to store a large amount of data, e.g., a multimedia program orpresentation.

Various I/O controllers are also connected to this system bus 621. TheseI/O controllers can include a keyboard controller 628, a mousecontroller 629, a video controller 630, and an audio controller 631. Asmight be expected, the keyboard controller 628 can provide the hardwareinterface for the keyboard 611, the mouse controller 629 can provide thehardware interface for mouse 612, the video controller 630 can providethe hardware interface for the display 660, and the audio controller 631can provide the hardware interface for the speakers 613, 614. AnotherI/O controller 633 can enable communication with the printer 615.

One of ordinary skill in the art will readily recognize that the PC 610can include a personal-digital-assistant (PDA), a laptop computer or avariety of other devices while remaining within the spirit and scope ofthe present invention.

The PC 610 may also be utilized in conjunction with a distributedcomputing environment where tasks are performed by remote processingdevices that are linked through a communications network The network mayinclude LANnets, a WANnets, the Internet and/or an Intranet. Clientterminals can include personal computers, stand-alone terminals, andorganizational computers. The stand-alone terminal may include hardwarefor loading smart cards, reading magnetic cards, and processingvideographics. The servers can include information servers,transactional servers and/or an external server. A transactional servermay perform financial and/or personal transactions. The network caninclude a graphical user interface for displaying a portion of thecharacteristic data on client terminals. The system may be operable witha plurality of third party applications. Additionally, the networks cancommunicate via wireless means or any of a variety of communicationmeans while remaining within the spirit and scope of the presentinvention.

Referring back to FIG. 6, an image captured by the image-capturingdevice 605 can be temporarily stored as image data in an image filewithin the image-capturing device 605. Accordingly, the orientationinformation can be stored in the image file as well. When the sensedimage is to be displayed by the PC 610, image-capturing device 605 andPC 610 are connected using the communication cable 620, and imageprocessing software installed on the PC 610 is started. Communicationsbetween the PC 610 and the image-capturing device 605 are done viacommunication software, and the image data stored in the image file istransmitted from the flash memory of the image-capturing device 605 tothe PC 610 via the communication cable 620.

The transmitted image data can then be temporarily stored in the harddisk 626 (see FIG. 7) of the PC 610. The image can then be transmittedfrom the hard disk 626 to the display 660 (see FIG. 7) for viewing bythe user. However, the remote image processing software 650 within thePC 610 reconfigures the image based on the orientation informationcontained in the image file. In an embodiment, the remote imageprocessing software 650 can retrieve the orientation information fromthe header of the transmitted image file. Alternatively, if the imagewas reconfigured utilizing local image processing software (i.e. imageprocessing software contained within the image-capturing device 605),the remote image processing software 650 simply displays the alreadyreconfigured image without further manipulation. Accordingly, the userviews the reconfigured image without having to account for a rotation ofthe image-capturing device 605 at the time the image was captured.

In an embodiment, when the image data transmitted from theimage-capturing device 605 is stored in the PC 605, the PC 605 starts aprinter driver for the printer 615, the image data captured from theimage-capturing device 605 is converted into print data that can beprinted by the printer 615 via the printer driver, and the convertedprint data is output to the printer 615 via the communication cable 625.The printer 615 receives the print data via the communication cable 625,and prints an image converted into the print data onto a print papersheet. In accordance with the present invention, the printer driverreconfigures the image based on the orientation information contained inthe image file and prints the reconfigured display of the image.

Although the above-described embodiment includes cable connections, oneof ordinary skill in the art will readily recognize that a variety ofconnections can be utilized. For example, a wireless connection, such asa Bluetooth radio link can be employed. Bluetooth is an open standardfor short-range transmission of digital voice and data between mobiledevices (laptops, PDAs, phones) and desktop devices. It supportspoint-to-point and multipoint applications. Unlike Infra-Red, whichrequires that devices be aimed at each other (line of sight), Bluetoothuses omni-directional radio waves that can transmit through walls andother non-metal barriers. Bluetooth transmits in the unlicensed 2.4 GHzband and uses a frequency hopping spread spectrum technique that changesits signal 1600 times per second. If there is interference from otherdevices, the transmission does not stop, but its speed is downgraded.

Additionally, in an embodiment of the present invention, the imageprocessing software can be configured to operate in a “re-orientation”mode whereby the image processing software actively searches the imagefile to find the orientation information of the image in order todetermine the orientation of the image-capturing device 605 with respectto a reference axis when the image is captured. Alternatively, the usercan disable this feature, thus allowing the image processing software tooperate in a “normal” mode whereby image files can be displayed withoutreconfiguring the image in order to account for a rotation of theimage-capturing device when the image was captured.

For a better understanding, please refer to FIG. 8. FIG. 8 is aflowchart of a method in accordance with an alternate embodiment of thepresent invention. First, an image is captured with an image-capturingdevice, via step 810. Next, the orientation of the image-capturingdevice is determined, via step 820. This can be accomplished with anorientation detection mechanism within the image-capturing device. Animage file is then created that includes the orientation of theimage-capturing device, via step 830. The image file is then transmittedto a system, via step 840. In an embodiment, the system is a personalcomputer. Finally, the system displays the image based on theorientation of the image-capturing device, via step 850. In anembodiment, image processing software within the personal computerreconfigures the display of the image based on the orientation of theimage-capturing device.

The above-described embodiments of the invention may also beimplemented, for example, by operating a computer system to execute asequence of machine-readable instructions. The instructions may residein various types of computer readable media. In this respect, anotheraspect of the present invention concerns a programmed product,comprising computer readable media tangibly embodying a program ofmachine readable instructions executable by a digital data processor toperform the method in accordance with an embodiment of the presentinvention.

This computer readable media may comprise, for example, RAM (not shown)contained within the system. Alternatively, the instructions may becontained in another computer readable media such as a magnetic datastorage diskette and directly or indirectly accessed by the computersystem. Whether contained in the computer system or elsewhere, theinstructions may be stored on a variety of machine readable storagemedia, such as a DASD storage (e.g. a conventional “hard drive” or aRAID array), magnetic tape, electronic read-only memory, an opticalstorage device (e.g., CD ROM, WORM, DVD, digital optical tape), paper“punch” cards, or other suitable computer readable media includingtransmission media such as digital, analog, and wireless communicationlinks. In an illustrative embodiment of the invention, themachine-readable instructions may comprise lines of compiled C, C++, orsimilar language code commonly used by those skilled in the programmingfor this type of application arts.

A method and system for processing an image has been disclosed.According to the present invention, a method and system is provided forreconfiguring a display of an image based on the orientation of theimage-capturing device when the image is captured. Through the use ofthe method and system in accordance with the present invention, a usercan view captured images without having to account for a rotation of theimage-capturing device at the time the image is captured.

Although the present invention has been described in accordance with theembodiments shown, one of ordinary skill in the art will readilyrecognize that there can be variations to the embodiments and thosevariations would be within the spirit and scope of the presentinvention.

Accordingly, many modifications may be made by one of ordinary skill inthe art without departing from the spirit and scope of the appendedclaims.

1-10. (canceled)
 11. An image processing system comprising: animage-capturing device capable of determining an orientation thereof inrelation to a reference axis; and a system coupled to theimage-capturing device wherein the system is capable of reconfiguring adisplay of an image captured by the image-capturing device based on theorientation.
 12. The image processing system of claim 11 wherein theimage-capturing device is capable of operating in a re-orientation mode.13. The image processing system of claim 11 wherein the image-capturingdevice includes an orientation detection mechanism.
 14. The imageprocessing system of claim 13 wherein the orientation detectionmechanism comprises an accelerometer.
 15. The image processing system ofclaim 13 wherein the image-capturing device comprises a digital camera.16. The image processing system of claim 13 wherein the image-capturingdevice comprises a mobile phone.
 17. The image processing system ofclaim 13 wherein the image-capturing device comprises a personal digitalassistant. 18-30. (canceled)